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Remarks 

This Request for Continued Examination and Reply is in response to the Final Office Action 
mailed June 9, 2009. 

I. Summary of Examiner's Rejections 

In the Office Action dated June 9, 2009, Claims 1-5, 9-14, 16-17 and 36-46 were rejected 
under 35 U.S.C. 103(a) as being unpatentable over Ludwig et al. (U.S. Patent No. 7,185,054, 
hereafter Ludwig) in view of Okuno (U.S. Patent No. 6,977,672) and further in view of Saka (U.S. 
Patent Publication No. 2004/0070608). Claims 6, 8, and 15 were rejected under 35 U.S.C. 103(a) 
as being unpatentable over Ludwig, in view of Okuno and Saka, and further in view of Shneiderman 
(U.S. Patent No. 7,010,751). Claim 7 was rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ludwig in view of Okuno and Saka, and further in view of Emens et al. (U.S. Patent No. 
6,463,343, hereafter Emens). Claims 18 and 21 were rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ludwig in view of Shneiderman. Claims 19-20 and 22-23 were rejected under 35 
U.S.C. 103(a) as being unpatentable over Ludwig in view of Shneiderman, and further in view of 
Okuno. Claims 24, 26-27, 30, 32, and 34-35 were rejected under 35 U.S.C. 103(a) as being 
unpatentable over Emens in view of Ayatsuka et al. (U.S. Patent No. 7,188,139, hereafter 
Ayatsuka), and further in view of Burt et al. ("Object tracking with a moving camera," IEEE An 
Application of Dynamic Motion Analysis, 1989, pp 2-12, hereafter Burt) and Shneiderman. Claim 25 
was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens in view of Ayatsuka, Burt, 
and Shneiderman, and further in view of Andersson (U.S. Patent Publication No. 2002/0111999 
A1). Claim 28 was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens in view of 
Ayatsuka, Burt, and Shneiderman, and further in view of Hildebrandt (U.S. Patent Publication No. 
2004/0070616). Claim 29 was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens 
in view of Ayatsuka, Burt, Shneiderman, and Andersson, and further in view of Westfield (U.S. 
Patent No. 6,677,979). Claim 31 was rejected under 35 U.S.C. 103(a) as being unpatentable over 
Emens in view of Ayatsuka, Burt, and Shneiderman, and further in view of Westfield. 

II. Summary of Applicant's Amendments 

The present Reply amends Claims 1, 10, 18, 24, 37 and 46; and cancels Claims 14-15, 
leaving for the Examiner's present consideration Claims 1-13, 16-32, 34-35, and 37-46. 
Reconsideration of the Application, as amended, is respectfully requested. 
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III. Claim Rejections Under 35 U.S.C. § 103 
Claims 1-17 and 36-46 

In the Office Action dated June 9, 2009, Claims 1-5, 9-14, 16-17 and 36-46 were rejected 
under 35 U.S.C. 103(a) as being unpatentable over Ludwig (U.S. Patent No. 7,185,054) in view of 
Okuno (U.S. Patent No. 6,977,672) and further in view of Saka (U.S. Patent Publication No. 
2004/0070608). Claims 6, 8, and 15 were rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ludwig, in view of Okuno and Saka, and further in view of Shneiderman (U.S. Patent No. 
7,010,751). Claim 7 was rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig in 
view of Okuno and Saka, and further in view of Emens (U.S. Patent No. 6,463,343). 

Claim 1 

Claim 1 as amended, recites: 

1. (Currently Amended) A method for exchanging information in a shared interactive 
environment, comprising: 

selecting a first remote physical device in a first live video image wherein the first 
remote physical device has information associated with it; 

causing the information to be transferred to a second remote physical device in a 
second live video image wherein the transfer is brought about by 

manipulating a visual representation of the information by interacting with the 
first live video image and the second live video image, 

querying the second remote physical device to determine if it can receive the 
information from the first remote physical device, and 

transferring the information from the first remote physical device to the 
second remote physical device; 

wherein at least one of the first remote physical device and the second remote 
physical device has a statically or dynamically defined hotspot in the first live video image or 
the second live video image; 

wherein the first remote physical device and the second remote physical device are 
part of the shared interactive environment; and 

wherein the first remote physical device and the second remote physical device are 
different remote physical devices. 

Ludwig discloses a teleconference system for conducting a teleconference among a plurality 
- 11 - 

Attorney Docket No.: FXPL-01063US0 
M:\nfeld\wp\FXPL\1063US0\1063US0_RespFOA_060909.doc 



Application No. 10/629,403 

Reply to Office Action dated: June 9, 2009 

Reply dated: October 8, 2009 

of participants. The system has a plurality of video display devices, each having associated 
participant video capture capabilities and participant audio capture and reproduction capabilities. 
(Abstract). 

Okuno discloses an information control system capable of executing a desired action in 
accordance with the angle position of a camera. (Abstract). Assume that panning, tilting, and 
zooming of the camera 115 are controlled, and a printer 21 1 connected to a network 201 is image- 
sensed as a main object by the camera 115, as shown in FIG. 18. In this case, as a program is 
activated, a window 38 pops up in the display screen 36 of the display unit 104, and the user can 
operate and set the printer 21 1 by using the window 37. (Column 4, lines 35-41 ). 

Saka discloses an apparatus and method for transferring files between a plurality of 
computers in a virtual network. (Abstract). Moreover, the present invention includes a more user- 
friendly display that allows a user to transfer a file between the two computers by simply moving the 
file icon from the second computer desktop to the adjacent computer desktop. (Paragraph [001 1]). 

Applicant respectfully submits that, based on the above description, Ludwig appears to 
disclose a teleconference system that enables a number of users to communicate with one another; 
Okuno appears to disclose a camera system that enables a user to control an object within view of 
the camera; and Saka appears to disclose a system that enables a user to transfer information 
between two computer desktops. 

Claim 1, as amended, recites transferring the information from the first remote physical 
device to the second remote physical device. While Okuno appears to disclose controlling a remote 
object and Saka appears to disclose transferring information from a local desktop to a remote 
desktop; Applicant respectfully submits that the cited references do not appear to disclose or render 
obvious transferring the information from the first remote physical device to the second remote 
physical device, as recited by Claim 1 . 

Claim 1, as amended, also recites querying the second remote physical device to determine 
if it can receive the information from the first remote physical device. Applicant respectfully submits 
that the cited references, when considered alone or in combination, do not disclose or render 
obvious this feature. 

In view of the above comments, Applicant respectfully submits that Claim 1, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
thereof is respectfully requested. 

Claims 10, 37, and 46 

The comments provided above with respect to Claim 1 are hereby incorporated by 
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reference. Claims 10, 37, and 46 have been similarly amended to more clearly recite the 
embodiments therein. For similar reasons as provided above with respect to Claim 1, Applicant 
respectfully submits that Claims 10, 37, and 46, as amended, are likewise neither anticipated by, 
nor obvious in view of the cited references, and reconsideration thereof is respectfully requested. 

Additionally, Claim 10 has been amended to recite annotating the first live video image; 
automatically transferring the annotation to the physical device if the annotation is at least partially 
drawn over the physical device as it appears in a live video image; and displaying the annotation on 
the physical device such that the annotation can be viewed at the remote location. Applicant 
respectfully submits that the cited references do not disclose or render obvious these features. 

Claims 2-9, 11-17, and 38-45 

Claims 14-15 have been canceled, rendering moot the rejection of these Claims. Claims 2- 
9, 1 1-17, and 38-45 depend from and include all of the features of Claims 1, 10, or 37. Claims 2-9, 
11-17, and 38-45 have not been addressed separately herein; however, Applicant respectfully 
submits that these claims are allowable at least as depending from an allowable independent claim, 
and further in view of the amendments to the independent claims, and the comments provided 
above. Reconsideration thereof is respectfully requested. 

Claims 18-23 

In the Office Action dated June 9, 2009, Claims 18 and 21 were rejected under 35 U.S.C. 
103(a) as being unpatentable over Ludwig in view of Shneiderman. Claims 19-20 and 22-23 were 
rejected under 35 U.S.C. 103(a) as being unpatentable over Ludwig in view of Shneiderman, and 
further in view of Okuno. 

Claim 18 

Claim 18 as amended, recites: 

18. (Currently Amended) A method for annotating a live video image, comprising: 

annotating the live video image, wherein the live video image shows a location including one 

or more physical devices at the location; 

automatically transferring the annotation to one of the physical devices at the location shown 

in the live video image if the annotation is at least partially drawn over the physical device as it 

appears in the live video image; 

displaying the annotation on the physical device such that the annotation can be viewed at 

the location; and 
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wherein the annotation is visible to at least one participant in a shared interactive 
environment. 

Ludwig discloses a teleconference system for conducting a teleconference among a plurality 
of participants. (Abstract). Ludwig also discloses that FIG. 2B illustrates data shared and 
annotated by those conferees (lower left window). (Column 6, lines 59-61; Figure 2B). 

Shneiderman discloses that [Shneiderman's] invention addresses the problem of annotating 
commercial and/or personal electronic images, by providing software that permits users to easily 
accomplish such annotation, through the drag-and-drop of annotations from a predefined, but 
extendable, list. The annotations are placed at an X,Y location on the image determined by the 
user, and stored in a searchable database. (Column 4, lines 23-29). Shneiderman also discloses 
that the term "electronic image," as used herein is intended to include photographs, video, 
drawings, writings, webpages, paintings, holographic images, cartoons, sketches, renderings, etc. 
that can be stored electronically. (Column 7, lines 29-32). 

Applicant respectfully submits that, based on the above description, Ludwig appears to 
disclose annotating shared data in a conference; and Shneiderman appears to disclose annotating 
electronic images. 

However, Claim 18, as amended, recites automatically transferring the annotation to one of 
the physical devices at the location shown in the live video image if the annotation is at least 
partially drawn over the physical device as it appears in the live video image. 

Applicant respectfully submits that Ludwig, in view of Shneiderman, does not appear to 
disclose or render obvious this feature. In both of the cited references, the annotation appears to 
be confined to the object being annotated. For example, the annotation in Ludwig appears to be 
limited to the graph shown in Fig. 2B and in Shneiderman the annotation appears to be limited to 
the particular image being annotated. Applicant respectfully submits that the cited references, 
when considered alone or in combination, do not disclose or render obvious, automatically 
transferring the annotation to one of the physical devices at the location shown in the live video 
image, as recited by Claim 18, as amended. 

Claim 18, as amended, also recites displaying the annotation on the physical device such 
that the annotation can be viewed at the location. 

Applicant respectfully submits that, as described above, the annotations shown and 
described by the cited references appear to be limited to the image being annotated and is not 
displayed on a physical device at the location shown in the live video image, such that the 
annotation can be viewed at the location. For example, the annotation of John Vince, shown in 
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Figure 7 of Shneiderman can only be viewed by someone looking at the annotated image, not by 
anyone located at the location shown in the image. Applicant respectfully submits that the cited 
references, when considered alone or in combination, do not disclose or render obvious, displaying 
the annotation on the physical device such that the annotation can be viewed at the location, as 
recited by Claim 18, as amended. 

In view of the above comments, Applicant respectfully submits that Claim 18, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
thereof is respectfully requested. 

Claims 19-23 

Claims 19-23 depend from and include all of the features of Claim 18. Claims 19-23 have 
not been addressed separately herein; however, Applicant respectfully submits that these claims 
are allowable at least as depending from an allowable independent claim, and further in view of the 
amendments to the independent claim, and the comments provided above. Reconsideration 
thereof is respectfully requested. 

Claims 24-35 

In the Office Action dated June 9, 2009, Claims 24, 26-27, 30, 32, and 34-35 were rejected 
under 35 U.S.C. 103(a) as being unpatentable over Emens in view of Ayatsuka (U.S. Patent No. 
7,188,139), and further in view of Burt ("Object tracking with a moving camera," IEEE An 
Application of Dynamic Motion Analysis, 1989, pp 2-12) and Shneiderman. Claim 25 was rejected 
under 35 U.S.C. 103(a) as being unpatentable over Emens in view of Ayatsuka, Burt, and 
Shneiderman, and further in view of Andersson (U.S. Patent Publication No. 2002/0111999 A1). 
Claim 28 was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens in view of 
Ayatsuka, Burt, and Shneiderman, and further in view of Hildebrandt (U.S. Patent Publication No. 
2004/0070616). Claim 29 was rejected under 35 U.S.C. 103(a) as being unpatentable over Emens 
in view of Ayatsuka, Burt, Shneiderman, and Andersson, and further in view of Westfield (U.S. 
Patent No. 6,677,979). Claim 31 was rejected under 35 U.S.C. 103(a) as being unpatentable over 
Emens in view of Ayatsuka, Burt, and Shneiderman, and further in view of Westfield. 

Claim 24 

Claim 24, as amended, recites: 

24. (Currently Amended) A shared interactive environment, comprising: 
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a camera system to provide a first live view and a second live view different from the 
first live view of a location, wherein the second live view can be configured to zoom in on a 
portion of the first live view; 

a first graphical user interface (GUI) coupled to the camera system and to present 
the first live view and the second live view, wherein the views can capture a physical device; 

a device controller to dynamically control the physical device in response to 
interaction of a first user with the GUI wherein the interaction can include annotating at least 
one of: 1) the first live view; and 2) the second live view; 

wherein annotations are automatically transferred to the physical device in the live 
views if the annotation is at least partially drawn over the physical device as it appears in the 
live video image, and wherein the annotation is displayed on the physical device such that 
the annotation can be viewed at the location; 

a device tracker coupled to the camera system and to dynamically recognize new 
physical devices; and 

wherein the camera system can be mounted on a mobile, robotic platform. 

Emens discloses a method for controlling a remote device from a client computer using a 
digital image of a remote location associated with the remote device. (Abstract). 

Ayatsuka discloses that each device arranged in an information space and a computer are 
sustainedly connected and the connection can be visually recognized on the display screen of the 
computer. (Abstract). 

Burt discloses that the task of detecting and tracking moving objects is particularly 
challenging if it must be performed with a camera that is itself moving. Yet, in applications such as 
automated surveillance and navigation, this task must be performed continuously, in real time, and 
using only modest computing hardware. (Abstract). 

Shneiderman is discussed in more detail with respect to Claim 18. 

As amended, Claim 24 recites that the annotation is displayed on the physical device such 
that the annotation can be viewed at the location. 

As discussed above with respect to Claim 18, while Shneiderman appears to disclose 
annotating electronic images, Applicant respectfully submits that Shneiderman does not disclose or 
render obvious that the annotation is displayed on the physical device such that the annotation can 
be viewed at the location, as recited in Claim 24. Applicant further respectfully submits that Emens, 
in view of Ayatsuka, Burt, and Shneiderman, when considered alone or in combination, does not 
disclose this feature. 

In view of the above comments, Applicant respectfully submits that Claim 24, as currently 
amended, is neither anticipated by nor obvious in view of the cited references, and reconsideration 
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thereof is respectfully requested. 

Claims 25-32 and 34-35 

Claims 25-32 and 34-35 depend from and include all of the features of Claim 24. Claims 25- 
32 and 34-35 have not been addressed separately herein; however, Applicant respectfully submits 
that these claims are allowable at least as depending from an allowable independent claim, and 
further in view of the amendments to the independent claim, and the comments provided above. 
Reconsideration thereof is respectfully requested. 

IV. Conclusion 

In view of the above amendments and remarks set forth above, it is respectfully submitted 
that all of the claims now pending in the subject patent application should be allowable, and 
reconsideration thereof is respectfully requested. The Examiner is respectfully requested to 
telephone the undersigned if he can assist in any way in expediting issuance of a patent. 

The Commissioner is authorized to charge any underpayment or credit any overpayment to 
Deposit Account No. 06-1325 for any matter in connection with this response, including any fee for 
extension of time, which may be required. 

Respectfully submitted, 



Date: October 8. 2009 By: /Nathan L Feld/ 

Nathan L. Feld 
Reg. No. 59,725 



Customer No. 23910 
FLIESLER MEYER LLP 
650 California Street, 14 th Floor 
San Francisco, California 94108 
Telephone: (415)362-3800 
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